Video captioning with stacked attention and semantic hard pull

Video captioning, i.e. , the task of generating captions from video sequences creates a bridge between Natural Language Processing and Computer Vision domains computer science. The semantically accurate description is quite complex. Considering complexity, problem, results obtained in recent research works are praiseworthy. However, there plenty scope for further investigation. This paper addresses this proposes novel solution. Most captioning models comprise two sequential/recurrent layers—one as video-to-context encoder other context-to-caption decoder. architecture, namely Semantically Sensible Captioning (SSVC) which modifies context generation mechanism by using approaches—“stacked attention” “spatial hard pull”. As no exclusive metrics evaluating models, we emphasize both quantitative qualitative analysis our model. Hence, have used BLEU scoring metric proposed human evaluation analysis, Semantic Sensibility (SS) metric. SS Score overcomes shortcomings common automated metrics. reports that use aforementioned novelties improves performance state-of-the-art architectures.